Designing planning and control interfaces to support user collaboration with flying robots

Robots are becoming increasingly prevalent and are already providing assistance in a variety of activities, ranging from space exploration to domestic housework. Recent advances in the design of sensors, motors, and microelectromechanical systems have enabled the development of a new class of small aerial robots. These free-flying robots hold great promise in assisting humans by acting as mobile sensor platforms to collect data in areas that are difficult to access or infeasible to instrument. In this work, we explored the design of interfaces that support users in working with free-flying robots to accomplish tasks including inventory logistics and management, environmental data collection, and visual inspection. Extending prior work in control interfaces for ground robots, we conducted a formative study in order to identify key design requirements for free-flyer interfaces. We designed several realistic tasks for use in evaluating human–robot interaction within the context of indoor free-flyer operation. We implemented three prototype interfaces that each provide varying degrees of support in enabling remote users to work with a flying robot to plan, communicate goals, accomplish tasks, and respond to changes in a dynamic environment. An experimental evaluation of each interface found that the interface designed to support collaborative planning and replanning using an interactive timeline and three-dimensional spatial waypoints significantly improved users’ efficiency in accomplishing tasks, their ability to intervene in response to spontaneous changes in task demands, and their ratings of the robot as a teammate compared to interfaces that support low-level teleoperation or waypoint-based supervisory control. Our results demonstrate the utility of a data-driven design process and show the need for free-flyer interfaces to consider planning phases in addition to task execution. In addition, we demonstrate the importance of providing interface support for interrupting robot operations as unplanned events arise.